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IN THE SPECIFICATION 

Please amend the Specification as follows: 

On pages 5-6, please delete the paragraph beginning on page 5, line 24 and ending on page 
6, line 6 and add the following paragraph at page 5, line 24: 

According to the invention, authors, i.e. users of the present invention, can review the 
creation and evolution of all or part of a content such as a multimedia title. An authoring 
tool incorporating the invention comprises a mechanism for recording the development of a 
content and a mechanism for replaying all or part of the recording. The recording 
mechanism records the steps performed during the development or modification of a 
content in the order in which the steps are performed. The recording mechanism preferably 
records the steps at a granularity, i.e. level of detail, which provides insight into the creative 
process and is useful for instructing authors on the development of the content. Desirably, 
the authoring tool records events at a scale where each action has a meaning, or represents 
an entire task or action. As part of the granularity, the recording mechanism is preferably 
designed to record important information, ignoring irrelevant activity. The recording 
mechanism may provide authors with only a single granularity throughout or it may vary 
between a plurality of granularities, selecting an appropriate granularity based on context 
and/or user-specified preference. 

On page 26, please delete the paragraph beginning at line 12 and ending at line 24 and add 
the following paragraph at page 26, line 12: 

As shown in FIG. 9, the playback module 212 comprises an initializing means 376 
for initializing the displayed software title for playback , means 378 for determining if at 
least one action remains to be played back, a selection means 380 for selecting which 
actions are to be played back and a playback means 382 for playing back one or more 
recorded actions. Optionally, the playback module further comprises a sorting means 384 
for sorting the recorded actions list and/or a redundancy removal means 386 for preventing 
the playback of redundant actions and for summarizing a sequence of redundant actions 
into a single action and replacing that single action for the redundant actions such that the 
single action is played back instead. Appendix B contains pseudocode for an example of 
redundant actions being summarized into a single action. 
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On page 26, please delete the paragraph beginning at line 26 and ending at line 35 and add 
the following paragraph at page 26, line 26: 

Preferably, the selecting means 380 allows an author to specify one or more criteria 
that actions generally must meet in order to be played back. Examples of criteria which can 
be specified include, but are not limited to, particular action(s), actions on particular 
object(s), and particular number of actions. For example, an author may want actions 
played back from the creation of a particular object or may want actions played back which 
relate to the creation of objects on a particular screen. Likewise, an author may want the last 
5 actions replayed or may want the first 10 actions replayed. 

IN THE CLAIMS 

Please delete Claims 1-18 without prejudice. 

Please add the following new Claims 19-27: 

19. (new) A method for recording and reviewing actions performed during 
development of software content created using a tool on a computer system having a 
processor and memory, said method comprising the steps of: 

receiving one or more user events; 

determining which events and sequences of events constitute actions; 
determining whether an explanation accompanies an action; 
recording the determined actions; and 

recording the determined explanations such that a recorded explanation of a 
recorded action is associated with the recorded action. 

20. (new) A method as defined in claim 33, said method further comprising the 
following steps: 

receiving a user request for playback of recorded actions; 

accessing recorded actions and associated recorded explanations; and 

playing back recorded actions and any associated recorded explanations. 
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21 . (new) A method as defined in claim 33, wherein said step of determining whether 
an explanation accompanies an action includes the step of prompting a user for an 
explanation with respect to an action being recorded, 

22. (new) A method for recording and reviewing actions performed during 
development of software content created using a tool on a computer system having a 
processor and memory, said method comprising the steps of: 

receiving one or more user events; 

determining which events and sequences of events constitute actions; 
recording the determined actions; 
receiving a user request for playback of recorded actions; 
accessing recorded actions; and 

playing back recorded actions, varying the levels of detail in the replay according to 
a current context in which the replay is occurring. 

23. (new) A computer-readable medium having stored thereon instructions for causing 
a computer to perform the following steps: 

displaying a starting state of multimedia content; 

determining automatically which recorded actions satisfy a specified arbitrary 
criteria; and 

playing back a sequence of only those determined recorded actions in chronological 
order on an output device. 

24. (new) A computer-readable medium having stored thereon instructions for causing 
a computer to perform the following steps: 

receiving a user event; 

determining whether said received user event indicates a playback request; and 
if said received user event indicates a playback request, then determining 
automatically which recorded actions to play back by determining which recorded 
actions meet a specified arbitrary criteria and playing back those determined 
recorded actions and if said received user event does not indicate a playback request, 
then determining whether there is an action to record and recording the action if it is 
determined that there is an action to record. 

25. (new) A computer-readable medium having stored thereon instructions for causing 
a computer to perform the following steps: 
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receiving a user event; 

determining whether said received user event indicates a playback request; and 
if said received user event indicates a playback request, then determining 
automatically which recorded actions to play back by determining which recorded 
actions meet a specified fixed criteria, said specified fixed criteria being selectable 
from a plurality of fixed criteria, and playing back those determined recorded actions 
and if said received user event does not indicate a playback request, then determining 
whether there is an action to record and recording the action if it is determined that 
there is an action to record, 

26. (new) A computer-readable medium having stored thereon an action class list 
comprising a plurality of action class description fields, each action class description field 
having a first field containing data which specifies a particular action class and a second 
field containing data which specifies a generic explanation of the action specified in the 
corresponding first field, the action class list arranged to be used during playback of an 
action to determine an explanation associated with the action class of the action and to 
accompany the played back action with the determined explanation. 

27. (new) A computer-readable medium as defined in claim 42 wherein said second 
field identifies a software routine capable of producing an explanation based upon 
properties of a recorded action. 
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REMARKS 

The present case is a continuation of U.S. Serial No. 09/482,745, filed January 13, 
2000 and due to issue on May 1, 2001 as U.S. Patent No. 6,226,785, which is a 
continuation of U.S. Serial No. 08/315,726, filed September 30, 1994, now abandoned. 
The original application was filed with claims 1-18. Applicant herein deletes claims 1-18 
and adds new claims 19 - 27. 

New claims 19 - 27 correspond to claims 33 - 36 and 39 - 43, respectively, in U.S. 
Patent No. 09/482,745. Applicants cancelled those claims without prejudice and are now 
reinstating those claims with some revisions as claims 19-27. The Examiner rejected 
claims 42 and 43 under 35 U.S.C. Section 101 as directed to non-statutory subject matter. 
The Examiner had rejected claims 39 - 41 under 35 U.S.C. Section 103(a) in view of U.S. 
Patent No. 5,157,779 (Washburn et al). The Examiner rejected claims 33 - 35 and 42 
under 35 U.S.C. Section 103(a) in view of U.S. Patent No. 5,524,193 (Covington et al). 

With regard to former claims 42 and 43, revised and newly numbered as claims 26 
and 27 in the present application, Applicants disagree with the Examiner's rejection of these 
claims under Section 101. A computer-readable medium is a statutory class of subject 
matter under 35 U.S.C. Section 101. In any regard, Applicants have revised claim 42 to 
read in-part "a first field containing data which specifies a particular action class and a 
second field containing data which specifies a generic explanation of the action specified in 
the corresponding first field". Applicants respectfully submit that claims 26 and 27 are 
statutory subject matter and satisfy Section 101. 

With regard to former claims 39-41, revised and newly-numbered as claims 23 - 
25, Applicants respectfully disagree with the Examiner's rejection under 35 U.S.C. Section 
103(a) in view of Washburn. 

Washburn et al. discloses a user extensible, automated testing system which 
provides for the recording and playback of user actions so that a user-defined test can be 
run on software in an identical fashion a repeated number of times. The Washburn testing 
system records events in a journal file, the events being representative of user actions taken 
to enable, initiate and execute certain tasks to test the computer system and/or software 
operating on the computer system, (col. 2, lines 64 - 68 and col. 5, lines 19 - 29 and lines 
53-61) The journal file is subsequently replayed to repeat the test whereby each event 
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recorded is read from the journal file and input to the window system in the sequence 
recorded. The window system responds to the events as if the events were received from the 
input devices, thereby executing the test, (col. 5, lines 19 - 29) 

The Washburn et al. system also includes a filter module for filtering out data which 
is not determinative of the pass or fail status of a test. To enable a filter function, the user, 
during the capture sequence, selects a menu item which causes the particular filter function 
or functions to be selected, (col. 7, lines 40 - 42) 

In contrast to the software testing system disclosed in Washburn et al., the present 
invention is directed to a system for the recording and replaying of actions performed 
during creation of software content such as, for example, multimedia content. In an aspect 
of the invention, the system is instructional, allowing review of the creation and evolution of 
all or part of multimedia content and providing insights into the creative process. The 
system includes, among other things, a recording mechanism and a playback mechanism. 

The recording mechanism records the steps performed during the development or 
modification of multimedia content, preferably recording the steps at a granularity, i.e. level 
of detail, which provides insight into the creative process and is useful for instructing 
authors on the development of the content. As part of the granularity, the recording 
mechanism is preferably designed to record important information, ignoring irrelevant 
activity. The recording mechanism may provide authors with only a single granularity 
throughout or it may vary between a plurality of granularities, selecting an appropriate 
granularity based on context and/or user-specified preference. 

The playback mechanism simulates the process of performing each step in the order 
in which it was recorded and, preferably, includes annotation using text, recorded or 
computer-generated voice, video and/or graphic animation. The level of details in the replay 
may vary according to the context in which the replay is occurring. For example, an event 
can described in some detail the first time it is explained and then later references to that 
event or similar events can be described in a more general manner. 

In an embodiment of creation replay an action class list specifies which action 
classes, i.e. categories of events or types of events, are recorded by the recording 
mechanism. For each action class, the action class list further specifies an explanation, 
verbal, text, video and/or animation, describing the particular action class. When an action is 
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played back, it is preferably accompanied by the explanation associated with the action class 
of the action. 

Preferably, an authoring tool embodying the invention allows an author to specify 
criteria for which actions are to be played back, thereby providing an author with a 
mechanism for focusing on a particular subset of the recorded actions. 

The present invention is significantly different and patentably distinct from that 
disclosed in Washburn for at least the reasons discussed below. First, as admitted by the 
Examiner, Washburn "does not disclose their product as a multimedia product." (Office 
Action, page 6). In contrast, the present invention provides for the development of 
multimedia content and allows the review of such development. 

Second, Washburn does not disclose recording and playing back events in such a 
way as to provide instruction and insight on the developmental process. Rather, the system 
in Washburn is directed to recording all events and playing them back in such a way as to 
provide repeated identical testing of software, (col. 5, line 66 -col. 6, line 4). The present 
invention, on the other hand, provides for recording and playing back of actions at different 
granularities so as to provide instruction and insight into the creative process of developing 
multimedia content. Authors are provided with a mechanism which they can use to 
understand the creative process that was used to create or modify a particular content such 
as a multimedia product. Understanding how content of others is created will usually help 
authors to advance their own content production skills or to modify the works of others, e,g< 
the multimedia product for which they are watching creation replay. End-users of 
multimedia products can use creation replay to learn about how the multimedia product was 
created, thereby gaining information on how to modify the product for their own individual 
uses. Additionally, by recording the events in the development process at a scale where each 
action has a meaning, or represents an entire task or operation, rather than recording each 
and every individual event, the amount of memory resources used to store the creation 
replay data is reduced. The use of memory resources is further minimized by pruning 
unimportant activities and periods of inactivity. 

Third, the present invention allows annotation along with the playback of the steps. 
Such annotation can supply authors with helpful information that supplements the 
simulation of the steps, thereby creating a more educational and instructive environment. 
Washburn et al. does not disclose such annotation. 
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Fourth, the present invention allows an author to specify criteria for the playback of 
the actions, so that the author can tailor the playback to meet his/her individual needs, 
thereby increasing the educational value and overall effectiveness of the invention. 
Washburn et al. does not provide such a mechanism. The filtering mechanism in 
Washburn et al. is directed to filtering of data to remove data which is not determinative of 
the pass or fail status of a test. Although Washburn et al. allows a user to manually edit the 
journal file, to add, edit or delete the recorded events, Washburn et al does not seem to 
disclose a mechanism for specifying criteria to automatically filter out which steps are 
played back, whereby steps which satisfy the specified criteria are filtered out by a playback 
or playback selecting mechanism before or during replay. Washburn et al. does not seem 
to disclose a system which allows arbitrary criteria to be specified. Moreover, Washburn et 
al. does not seem to disclose a system which matches recorded actions with specified 
arbitrary criteria in order to determine which recorded actions to replay. Nor does 
Washburn et al. seem to disclose a system which matches recorded actions with specified 
criteria in order to determine which recorded actions to replay, the determined actions not 
necessarily being sequential. These differences, among others, are discussed below with 
respect to the Claims. 

Regarding, claim 23, although Washburn et al. allows a user to manually edit the 
journal file, manually adding, editing or deleting the recorded events, Washburn et al. does 
not disclose a mechanism for specifying arbitrary criteria to automatically determine which 
steps are played back, whereby steps which satisfy the specified criteria are determined by a 
playback or playback selecting mechanism before or during replay. Claim 23 specifically 
recites in part 11 determining automatically which recorded actions satisfy a specified 
arbitrary criteria;" and "playing back a sequence of only those determined recorded actions 
in chronological order on an output device 1 '. At col. 20, lines 18 - 24, Washburn discusses 
that a user can add information in the journal file. As an example, the user may select the 
add function and manually perform user actions to perform a bit map save. This is 
drastically different from the claimed invention in which it is automatically determined 
which recorded actions satisfy a specified arbitrary criteria and only those recorded actions 
are played back. Therefore, claim 23 is patentably distinct from Washburn and should be 
allowed. 

Claim 24 is likewise patentable over Washburn for at least the reasons discussed 
relative to claim 23. The Examiner cited Washburn at col. 20, lines 3 - 4 as disclosing 
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"determining which recorded action to play back". Applicants respectfully disagree with the 
Examiner's interpretation of this portion of Washburn. At col, 19, line 65 - col. 20, line 8, 
Washburn discusses a method for manually editing the journal file. A user may delete, add 
or otherwise change an event by selecting the desired script. The high level script is 
transformed back into the binary form and the journal file is updated. This is significantly 
different from claim 24 wherein if a playback request is indicated, then it is determined 
automatically which recorded actions to play back by determining which recorded actions 
meet a specified arbitrary criteria and playing back those determined recorded actions. 
Therefore, claim 24 is patentably distinct from Washburn. 

Claim 25 is also patentable over Washburn for at least the reasons discussed 
previously in regard to claims 23 and 24. The Examiner specifically cites Washburn at col. 
20, lines 27 as disclosing "fixed criteria". However, that section of Washburn discusses 
deleting events, not playing back events according to a fixed criteria. Therefore, claim 25 is 
patentable over Washburn. 

The Examiner rejected claims 33 - 35 and 42 under 35 U.S.C. Section 103(a) in 
view of U.S. Patent No. 5,524,193 (Covington et al.). These claims correspond to new 
claims 19 - 21 and 26. Applicants respectfully disagree with the Examiner's rejection of 
these claims and believe that these claims are patentable over Covington for at least the 
following reasons. 

Covington et al. generally describes a method for annotating a text document or 
other media event. It allows authors to create a sequence of media events that are connected 
to a particular word or phrase (trigger) in a text document. (Covington at col. 2, lines 42 - 
48) A trigger, and the sequence of media events connected to that trigger, can be associated 
with a particular filter, i.e. a particular grouping of triggers. (Covington at col 2, lines 60 - 
64) A connection map, i.e. a look-up table in the computer's memory, records the location 
of triggers, the filter with which each trigger is associated, and the media sequence 
connected to that trigger. (Covington at col. 3, lines 8-11) Users can read and explore 
work by using the Explore Mode of the media manager. (Covington at col. 7, lines 1-9) 
Triggers and event sequences can be created and edited in Analyze mode, while the Create 
mode can be used to add text to and edit the displayed work or to create a new document. 
(Covington at col. 7, lines 1 1 - 25) 
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The present invention is patentably distinct from Covington for at least the following 
reasons. Claim 19 states in part "determining which events and sequences of events 
constitute actions". Although Covington discusses that "a word, phrase or section to which 
a sequence of media events is connected will be referred to herein as a 'trigger'" (Covington 
at col. 2, lines 46 - 48), Covington does not disclose, teach or otherwise suggest determining 
which events and sequences of events constitute actions. Rather than recording all events, 
the present invention as defined in claim 19 determines which events and sequences of 
events constitute actions and records those determined actions. 

Additionally, Covington does not disclose "recording the determined explanations 
such that a recorded explanation of a recorded action is associated with the recorded action" 
as recited in claim 19. The Examiner cited Covington at col. 3, lines 8 - 1 1 as disclosing 
this claim element, stating that the trigger is similar to the explanation. Applicants 
respectfully disagree. Covington defines a trigger as "the word, phrase or section to which a 
sequence of media events is connected" (col. 2, lines 46 - 48). Media events connected to a 
highlighted trigger may be accessed by selecting that trigger, (col. 5, lines 33 - 34). This is 
different from the claimed invention in which a recorded explanation of a recorded action is 
associated with the recorded action. Therefore, Applicants respectfully assert that claim 19, 
and its dependent claims 20 and 21, are patentable over Covington. 

Regarding claim 26, the Examiner equates a connection map in Covington with an 
action class, a filter in Covington with specifying an action class and a trigger in Covington 
with the generic explanation of the action. However, the connection map specifies the 
location of a trigger and the filter it is associated with, as well as the media sequence 
connected to the trigger. (Covington at col. 2, lines 8-11) A trigger in Covington is a 
word, phrase or section to which a sequence of media events is connected. (Covington at 
col. 2, lines 46 - 48) The filter and trigger together determine which media sequence is 
played. (Covington at col. 5, lines 11-16) 

An action class list, on the other hand, specifies action classes, where an action class 
is a category of events or type of events. (Specification, page 6, lines 17 - 19). An event is a 
low-level, common occurrence such as a cursor movement, pressing or releasing a mouse 
button or key, inserting a disk, or turning a dial. (Specification, page 15, lines 31 - 34) An 
action is a goal-directed set of events which has an effect or consequence on the software 
title or content, thereby progressing the development of the software title or content. 
(Specification, page 16, lines 3) An action class list associates an action class with a generic 
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explanation such that when an action belonging to that particular action class is played back, 
it is accompanied by the generic explanation associated with that action class. 

Covington does not teach, disclose or otherwise suggest, aggregating events into 
actions and categorizing actions into action classes such that an action class has an 
associated generic explanation. For at least the reasons discussed above, Applicants 
respectfully request allowance of this claim, as well as dependent claim 27. 

In conclusion, Applicants respectfully assert that claims 19 - 27 are patentable and in 
condition for allowance. In light of the discussion above, Applicants respectfully request 
allowance of such claims. 



Respectfully submitted, 
APPLE COMPUTER, INC. 



Date: April 26. 2001 
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MARKED UP VERSIONS OF AMENDMENTS 
IN THE SPECIFICATION: 

Please delete the paragraph beginning at page 5, line 24, and ending at page 6, line 6 and add 
the following paragraph: 

According to the invention, authors, i.e. users of the present invention, can review the 
creation and evolution of all or part of a content such as a multimedia title. An authoring 
tool incorporating the invention comprises a mechanism for recording the development of a 
content and a mechanism for replaying all or part of the recording. The recording 
mechanism records the steps performed during the development or modification of a 
content in the order in which the steps are performed. The recording mechanism preferably 
records the steps at a granularity, i.e, level of detail, which provides insight into the creative 
process and is useful for instructing authors on the development of toe content. Desirably, 
the authoring tool records events at a scale where each action has a meaning, or represents 
an entire task or action. As part of the granularity, the recording mechanism is preferably 
designed to record important information, ignoring irrelevant activity. The recording 
mechanism may provide authors with only a single granularity throughout or it may vary 
between a plurality of granularities, selecting an appropriate granularity based on context 
and/or user[ ] z specified[-]„preference. 

On page 26, please delete the paragraph beginning at line 12 and ending at line 24 and add 
the following paragraph at page 26, line 12: 

As shown in FIG, 9, the playback module 212 comprises an initializing means 376 
for initializing the displayed software title for playback , means 378 for determining if at 
least one action remains to be played back, a selection means 380 for selecting which 
actions are to be played back and a playback means 382 for playing back one or more 
recorded actions. Optionally, the playback module further comprises a sorting means 384 
for sorting the recorded actions list and/or a redundancy removal means 386 for preventing 
the playback of redundant actions and for summarizing a sequence of redundant actions 
into a single action and replacing that single action for the redundan[g]t actions such that the 
single action is played back instead. Appendix B contains pseudocode for an example of 
redundant actions being summarized into a single action. 
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On page 26, please delete the paragraph beginning at line 26 and ending at line 35 and add 
the following paragraph at page 26, line 26: 

Preferably, the selecting means 380 allows an author to specify one or more criteria 
that actions generally must meet in order to be played back. Examples of criteria which can 
be specified include, but are not limited to, particular action(s), actions on particular 
object(s), and particular number of actions. For example, [a] an author may want actions 
played back from the creation of a particular object or may want actions played back which 
relate to the creation of objects on a particular screen. Likewise, an author may want the last 
5 actions replayed or may want the first 10 actions replayed. 
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